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DETAILED ACTION 

Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1-9, 1 1 , 13, 16, 19-25, 31, and 33 rejected under 35 U.S.C. 103(a) as 
being unpatentable overTsuchikawa; Megumu et al. US 5748775 A (hereinafter 
Tsuchikawa) in view of Soundararajan, Aravind US 20030106072 A1 (hereinafter 
Soundararajan). 

Re claims 1 and 19, Tsuchikawa teaches an apparatus for event-driven content 
analysis, within a computerized system having a processing unit and a storage unit 
(Col. 2 line 55 - Col. 3 line 13), the apparatus comprising the elements of: 

a pivot spot definer component to mark an at least one position in the interaction 
media to indicate the occurrence of an at least one pre-defined event or data item (Col. 
5 lines 18-26); 

a region of interest definer component to determine the limits of an at least one 
segment of the interaction associated with the location of the pivot spot (Col. 5 lines 27- 
51) 
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However, Tsuchikawa fails to teach a media type selector component to select a 
type of an interaction media inputted for analysis from an at least one interaction 
recording or storage device (Soundararajan [0019]); 

Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention a media type selector to select a type of interaction media inputted 
for analysis for the purposes of a universal control system fro various multimedia 
equipment/data, wherein all multimedia can interface through a single distributed 
storage media. 
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Re claim 2, Tsuchikawa teaches the apparatus of claim 1 further comprising a 
content analysis input selector component to determine an at least one input or 
parameter for an at least one analyzer component (Col. 5 lines 5-1 7). 

Re claim 3, Tsuchikawa teaches the apparatus of claim 1 further comprises an 
analysis type selector component to identify and to select an at least one analyzer 
component type for determining the Region of Interest (Col. 5 lines 27-51 ). 

Re claim 4, Tsuchikawa teaches the apparatus of claim 1 further comprising an 
audio analyzer component for performing an analysis on the media selected by the 
media selector component in a location adjacent to the pivot spot identified by the pivot 
spot defined component (Col. 5 lines 5-17). 

However, Tsuchikawa fails to teach a media selector component (Soundararajan 
[0019]); 

Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
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include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention a media type selector to select a type of interaction media inputted 
for analysis on a frame by frame basis for the purposes of a universal control system fro 
various multimedia equipment/data, wherein all multimedia can interface through a 
single distributed storage media. 

Re claim 5, Tsuchikawa teaches the apparatus of claim 1 further comprising an 
analyzer component for performing an analysis on the media selected by the media 
selector component in a location adjacent to the pivot spot identified by the pivot spot 
definer component (Col. 5 lines 5-17) using an analyzer selected by the analysis type 
selector using parameters fed to, or selected by the content analysis input selector 
component (Col. 5 lines 27-51). 

However, Tsuchikawa fails to teach a media selector component (Soundararajan 
[0019]); 

Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
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National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention a media type selector to select a type of interaction media inputted 
for analysis on a frame by frame basis for the purposes of a universal control system fro 
various multimedia equipment/data, wherein all multimedia can interface through a 
single distributed storage media. 

Re claim 6, Tsuchikawa teaches the apparatus of claim 1 further comprising an 
analyzer component for performing an analysis on the media selected by the media 
selector component within the region of interest identified by the region of interest 
definer component (Col. 5 lines 27-51 ) using an analyzer selected by the analysis type 
selector using parameters fed to (Col. 5 lines 5-17), or selected by the content analysis 
input selector component 

However, Tsuchikawa fails to teach a media selector component (Soundararajan 
[0019]); 

Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
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intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention a media type selector to select a type of interaction media inputted 
for analysis on a frame by frame basis for the purposes of a universal control system fro 
various multimedia equipment/data, wherein all multimedia can interface through a 
single distributed storage media. 

Re claims 7 and 31 , Tsuchikawa teaches the apparatus of claim 1 wherein the 
region of interest defined by the region of interest definer component further comprises 
an optimization component for optimizing the region of interest (Col. 5 lines 27-51 & Fig. 
3 item 200) 
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Re claim 8, Tsuchikawa teaches an apparatus of claim 1 further comprises a 
content analysis inputs table to hold in storage the at least one selectable input values 
(Col. 5 lines 5-17). 

Re claim 9, Tsuchikawa teaches the apparatus of claim 1 further comprises the 
element of an audio analyzer component to analyze the audio elements of the 
interaction data (Col. 5 lines 5-17). 

Re claim 1 1 , Tsuchikawa teaches the apparatus of claim 1 further comprises a 
screen event analyzer component to identify and capture an at least one screen and an 
at least one screen event associated with the interaction data (Col. 5 lines 5-17 & Fig. 
4). 

Re claim 13, Tsuchikawa teaches the apparatus of claim 1 further comprising an 
analysis module for performing an analysis on the media (Col. 5 lines 5-17 & Fig. 4). 

Re claim 16, Tsuchikawa teaches the apparatus of claim 1 wherein the region of 
interest is a specific segment of the interaction media that is analyzed to extract 
meaningful interaction-specific information in an organization (Col. 5 lines 27-51). 

Re claim 20, Tsuchikawa fails to teach selecting an interaction media to analyze 
(Soundararajan [0019]); 
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Soundararajan teaches a multimedia source input for receiving multimedia 
content from a multimedia stream 102. The term "multimedia stream" as used herein is 
intended to refer to signals that may originate from a cable, satellite, or terrestrial 
broadcast, or from an alternative multiple-program stream. The multimedia content 
preferably includes, but is not limited to, broadcast television video signals, such as 
National Television Standards Committee (NTSC) signals, DTV signals, and high 
definition television (HDTV) signals. The multimedia content may also include still 
images, audio signals (e.g., from a satellite radio source), etc. The STB/DTV 104 may 
include multiple multimedia inputs and a multiplexer or equivalent switching circuitry (not 
shown) for selecting one of a plurality of multimedia sources (e.g., video camera, digital 
still camera, etc.). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention selecting an interaction media to analyze for the purposes of a 
universal control system fro various multimedia equipment/data, wherein all multimedia 
can interface through a single distributed storage media. 

Re claim 21 , Tsuchikawa teaches the method of claim 19 further comprising the 
step of selecting a method for the analysis of the at least one interaction media based 
on the at least one event associated with the interaction (Fig. 3). 
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Re claim 22, Tsuchikawa teaches the method of claim 19 further comprising the 
step of selecting a method for the analysis of the at least one interaction media based 
on the result of a previously performed analysis (Fig. 6 item 130) 

Re claim 23, Tsuchikawa teaches the method of claim 19 further comprising the 
step of selecting the parameters to be used in the at least one analysis instruction step 
on the at least one segment of the interaction media (Col. 5 lines 5-17). 

Re claim 24, Tsuchikawa teaches the method of claim 19 further comprising the 
step of optimizing the region of interest by performing an at least one analysis 
instruction step within the region of interest (Col. 5 lines 27-51) and readjusting the 
region of interest in accordance with the results of the at least one analysis instruction 
step (Fig. 3). 

Re claim 25, Tsuchikawa teaches the method of claim 19 wherein the region of 
interest is predetermined by a user or an apparatus (Col. 5 lines 27-51). 

Re claim 33, Tsuchikawa teaches the method of claim 19 further comprising the 
step of adjusting the at least one pivot spot (Fig. 4) or region of interest on the 
interaction media (Col. 5 lines 27-51). 
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3. Claims 10, 14, 15, 17, 18, 26, 30, and 32 rejected under 35 U.S.C. 103(a) as 
being unpatentable over Tsuchikawa; Megumu et al. US 5748775 A (hereinafter 
Tsuchikawa) in view of Soundararajan, Aravind US 20030106072 A1 (hereinafter 
Soundararajan) and further in view of Kung; Fen-Chung et al. US 6917610 B1 
(hereinafter Kung). 

Re claim 10, Tsuchikawa in view of Soundararajan fails to teach the apparatus of 
claim 1 further comprises a computer telephony interface events analyzer component to 
identify and capture at least one common telephony events associated with the 
interaction data (Kung Col. 33 lines 1-25) 

Kung teaches an activity log may indicate using easy to understand icons as to 
whether the communication is a telephone calls (e.g., record 831), a multimedia video 
and audio calls (e.g., records 832 and 841), a text email (e.g., record 842), and/or an 
instant message email (e.g., record 840), etc. Further, the activity log GUI may be 
include a feature button 802 with a pull down menu that allows the subscriber to select 
between combined media record listing, or one or more of the particular media types 
such as telephone calls, multimedia video and audio calls, text emails, instant message 
email, etc., to be displayed on the GUI at the same time. In addition, the activity log 
may include records, such as record 833, of incoming communications which have no 
known DN or address associated with them, when the originating party uses, for 
example, a feature such as caller ID blocking. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention a computer telephony interface to identify and capture common 
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telephony events to allow for a restricted list of subscribers that only have access to 
phone information (i.e. Caller ID blocking). 

Re claim 14, Tsuchikawa in view of Soundararajan fails to teach the apparatus of 
claim 1 wherein the interaction is one of the following: a telephone call, and e-mail 
message, an audio recording, a video, multimedia data or an interaction media (Kung 
Col. 33 lines 1-25). 

Kung teaches an activity log may indicate using easy to understand icons as to 
whether the communication is a telephone calls (e.g., record 831), a multimedia video 
and audio calls (e.g., records 832 and 841), a text email (e.g., record 842), and/or an 
instant message email (e.g., record 840), etc. Further, the activity log GUI may be 
include a feature button 802 with a pull down menu that allows the subscriber to select 
between combined media record listing, or one or more of the particular media types 
such as telephone calls, multimedia video and audio calls, text emails, instant message 
email, etc., to be displayed on the GUI at the same time. In addition, the activity log 
may include records, such as record 833, of incoming communications which have no 
known DN or address associated with them, when the originating party uses, for 
example, a feature such as caller ID blocking. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention interaction media inputted for analysis for the purposes of a 
universal control system fro various multimedia equipment/data, wherein all multimedia 
can interface through a single distributed storage media. 
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Re claim 15, Tsuchikawa in view of Soundararajan fails to teach the apparatus of 
claim 14 wherein the interaction media is at least one data packet carrying voice or 
other media over internet protocol (Kung Col. 7 lines 19-33). 

Kung teaches an IP central station 200 may be configured to provide connectivity 
for the broadband residential gateway 300 to the Internet 180 (e.g., World Wide Web 
(www)), as well as connectivity to other external networks such as public switched 
telephone network 160 and signaling system 7 (SS7) 170 for end-to-end voice, 
multimedia, and data applications, for example voice over IP telephony. IP packets 
traveling through the IP network provide for priority so that, for example, voice packets 
are given priority over data packets to maintain certain VoIP telephony QoS 
requirements and a leased line concept for packet traffic which may have an even 
higher priority. However, the system is sufficiently flexible so that the priority can be 
dynamically altered according to customer preferences, variable billing rates, traffic 
patterns, and/or congestion. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention data packets carrying voice over IP to allow for the maintenance of 
telephony operations to assign priority for transmission data, wherein various device 
can be used communicate simultaneously (i.e. phone, PC, etc.). 
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Re claim 17, Tsuchikawa in view of Soundararajan fails to teach the apparatus of 
claim 1 wherein the interaction meta-data (Kung Col. 33 lines 26-39) is an at least one 
computer telephony integrated (Kung Col. 33 lines 1-25) or CRM event. 

Kung teaches an activity log GUI may have a default set of categories which will 
be displayed to the subscriber, e.g., which determines the type of information such as 
the DN, a system address, an email address 808, a "contacted" party's name 809, a 
company name 810, a client billing number 81 1 , as well as billing information such as 
the date 812, time 813, length of the previous communication (duration 815) and any 
associated charges (cost 815) for the previous communication. Further, the GUI may 
include a Category button 804 which enables the user to alter the information displayed 
on the GUI and/or retained in the activity log. Thus, for example the user may alter the 
activity log to include such as information as a contacted party's street address or 
include alternative DN's or system/email address. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention metadata to allow for telephony or audio communication, wherein 
information about a conversation or video can be retained to allow for indexing of 
recognizable information such as a pattern or sequence of repetitive recorded 
characteristics. 

Re claims 18 and 26, Tsuchikawa in view of Soundararajan fails to teach the 
apparatus of claim 1 wherein the interaction meta-data is associated with the at least 
one screen event (Kung Col. 33 lines 26-39). 
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Kung teaches an activity log GUI may have a default set of categories which will 
be displayed to the subscriber, e.g., which determines the type of information such as 
the DN, a system address, an email address 808, a "contacted" party's name 809, a 
company name 810, a client billing number 81 1 , as well as billing information such as 
the date 812, time 813, length of the previous communication (duration 815) and any 
associated charges (cost 815) for the previous communication. Further, the GUI may 
include a Category button 804 which enables the user to alter the information displayed 
on the GUI and/or retained in the activity log. Thus, for example the user may alter the 
activity log to include such as information as a contacted party's street address or 
include alternative DN's or system/email address. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention metadata to allow for telephony or audio communication with the 
aid of visually seeing past data, wherein information about a conversation or video can 
be retained to allow for indexing of recognizable information such as a pattern or 
sequence of repetitive recorded characteristics. 

Re claim 30, Tsuchikawa in view of Soundararajan fails to teach the method of 
claim 19 further comprises the steps of: 

identifying an at least one pre-defined computer telephony integrated event in the 
interaction data (Kung Col. 33 lines 1-25); 

identifying an at least one pre-defined screen event in the interaction data (Kung 
Col. 33 lines 26-39) 
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Kung teaches an activity log GUI may have a default set of categories which will 
be displayed to the subscriber, e.g., which determines the type of information such as 
the DN, a system address, an email address 808, a "contacted" party's name 809, a 
company name 810, a client billing number 81 1 , as well as billing information such as 
the date 812, time 813, length of the previous communication (duration 815) and any 
associated charges (cost 815) for the previous communication. Further, the GUI may 
include a Category button 804 which enables the user to alter the information displayed 
on the GUI and/or retained in the activity log. Thus, for example the user may alter the 
activity log to include such as information as a contacted party's street address or 
include alternative DN's or system/email address. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention metadata to allow for telephony or audio communication with the 
aid of visually seeing past data, wherein information about a conversation or video can 
be retained to allow for indexing of recognizable information such as a pattern or 
sequence of repetitive recorded characteristics. 

Re claim 32, Tsuchikawa in view of Soundararajan fails to teach the method of 
claim 19 further comprises performing an at least one content analysis step during the 
capturing of the interaction data and the interaction meta-data (Kung Col. 33 lines 26- 
39) 

Kung teaches an activity log GUI may have a default set of categories which will 
be displayed to the subscriber, e.g., which determines the type of information such as 
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the DN, a system address, an email address 808, a "contacted" party's name 809, a 
company name 810, a client billing number 81 1 , as well as billing information such as 
the date 812, time 813, length of the previous communication (duration 815) and any 
associated charges (cost 815) for the previous communication. Further, the GUI may 
include a Category button 804 which enables the user to alter the information displayed 
on the GUI and/or retained in the activity log. Thus, for example the user may alter the 
activity log to include such as information as a contacted party's street address or 
include alternative DN's or system/email address. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention metadata to allow for telephony or audio communication with the 
aid of visually seeing past data, wherein information about a conversation or video can 
be retained to allow for indexing of recognizable information such as a pattern or 
sequence of repetitive recorded characteristics. 

4. Claims 12 and 27-29 rejected under 35 U.S.C. 103(a) as being unpatentable 
over Tsuchikawa; Megumu et al. US 5748775 A (hereinafter Tsuchikawa) in view 
of Soundararajan, Aravind US 20030106072 A1 (hereinafter Soundararajan) and 
further in view of Chase; Wayne O. US 6332143 B1 (hereinafter Chase). 

Re claims 12 and 27-29, Tsuchikawa in view of Soundararajan fails to teach the 
apparatus of claim 4, wherein the audio analyzer component further comprises the 
elements of: 
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a word spotting component to locate and identify pre-defined terms or patterns in 
the speech elements of the interaction data (Chase Col. 17 lines 50-63); 

an emotion analysis component to locate and identify positive or negative 
emotions in the interaction data (Chase Col. 11 lines 45-53); 

a talk analyzer component to identify and locate specific pre-defined speech 
events in the speech elements of the information data (Chase Col. 17 lines 50-63 & Fig. 
5 item 57) 

Chase teaches an emotional descriptor that most closely matches the emotional 
connotation that the connotative judge associates with the word or phrase, considering 
the denotative context and part of speech, wherein a connotative judge understands the 
word or phrase and its denotative context, but does not associate any of the emotional 
descriptors from the supplied list of emotional descriptors with the word or phrase and 
its denotative context. Further, Chase teaches high level information of FIG. 5 includes 
a rating of the passage for emotional content in terms of positive emotion, negative 
emotion and global emotion (i.e., positive not distinct from negative emotion). The high 
level information also includes a rating of the passage for human interest by words, 
sentences and global content, a rating of the passage for power, a rating of the 
passage for activity and a rating of the passage along a scale of abstractness versus 
concreteness. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention an audio analyzer to spot words and negative/positive emotions 
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associated with speech data for the purposes of understanding the context of a situation 
if there is no video available. 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. US 6426771 B1 , US 4351007 A, US 5922047 A, US 
20050132293 A1 , US 20050238000 A1 , US 20050204378 A1 . 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael C. Colucci whose telephone number is (571)- 
270-1847. The examiner can normally be reached on 9:30 am - 6:00 pm, Monday- 
Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (571)-272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Michael Colucci Jr. 
Patent Examiner 
AU 2626 
(571)-270-1847 

Michael.Co lucci@uspto.qQv 

/Richemond Dorvil/ 

Supervisory Patent Examiner, Art Unit 2626 



